Thanks. Though one thing to note is that the actual speech_commands dataset by itself doesn't match what the AST-v2 model was fine-tuned on, and you probably need to do some pre-processing to remove extra labels: huggingface/datasets#6446

Also I think you need to query file from the examples, currently getting an error from here:

  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/olive/data/component/pre_process_data.py", line 259, in _tokenizer_and_align_labels
    for files in tokenized_inputs[kwargs.get("file_column_name", "file")]:
  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/transformers/feature_extraction_utils.py", line 86, in __getitem__
    return self.data[item]
KeyError: 'file'

Thanks. Though one thing to note is that the actual speech_commands dataset by itself doesn't match what the AST-v2 model was fine-tuned on, and you probably need to do some pre-processing to remove extra labels: huggingface/datasets#6446

Yes, I manually align the labels by file prefix. The way you show is the better way. Updated.

Also I think you need to query file from the examples, currently getting an error from here:

  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/olive/data/component/pre_process_data.py", line 259, in _tokenizer_and_align_labels
    for files in tokenized_inputs[kwargs.get("file_column_name", "file")]:
  File "/Users/victor/anaconda3/envs/transformers-v2/lib/python3.9/site-packages/transformers/feature_extraction_utils.py", line 86, in __getitem__
    return self.data[item]
KeyError: 'file'

fixed.

[Not merge] Let us wait for customer's feedback.

From the linked issue, you probably need to remove the _silence_ label before running align_labels_with_mapping, currently getting a _silence_ KeyError. Not very clean but maybe you can make this an option.

/azp run

From the linked issue, you probably need to remove the _silence_ label before running align_labels_with_mapping, currently getting a _silence_ KeyError. Not very clean but maybe you can make this an option.

https://github.com/microsoft/Olive/pull/762/files#:~:text=%23%20align%20labels%20with,%5B0%5D)
I already did that. The config example worked for me. When you trigger the error of _silence_ key error? Have you updated your config json with the latest one in this pr? @vymao

latest changes for json config:
https://github.com/microsoft/Olive/pull/762/files#diff-f9735dfb6fa445feeb666b7f11c9b670283b799fecf876683962741bee8733fe:~:text=%22component_kwargs%22%3A%20%7B,%7D

Thanks, works.